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DETAILED ACTION 

1. This action is responsive to communications: amendment filed 6/18/2003, to the original application 
filed 2/6/2002, drawings filed 3/25/2002. The present application is a continuation (filed under 37 CFR 1.53(b)) 
of parent case 08/900,421 (now abandoned). Said parent case filed 7/25/1997. 

2. The rejection of claims 1,9, 16 under 35 U.S.C. 102(e) as being anticipated by Katseff has been 
withdrawn as necessitated by amendment. ... _ 

3. The rejection of claims 2, 4, 12, 14, 18 under 35 U.S.C. 103(a) as being unpatentable over Katseff, and 
Shibata, has been withdrawn as necessitated by amendment. 

4. The rejection of claim 3 under 35 U.S.C. 103(a) as being unpatentable over Katseff, and Baumgartner, 
has been withdrawn as necessitated by amendment. 

5. The rejection of claims 5, 13 under 35 U.S.C. 103(a) as being unpatentable over Katseff, has been 
withdrawn as necessitated by amendment. 

6. The rejection of claims 6, 7, 10, 17 under 35 U.S.C. 103(a) as being unpatentable over Katseff, and 
Durward, has been withdrawn as necessitated by amendment. 

7. Claim 19 remains rejected under 35 U.S.C. 103(a) as being unpatentable over Katseff, and Durward. 

8. The rejection of claims 8, 1 1 under 35 U.S.C. 103(a) as being unpatentable over Katseff, and 
Wasserman, has been withdrawn as necessitated by amendment. 

9. The rejection of claim 20 under 35 U.S.C. 103(a) as being unpatentable over Katseff, Durward, and 
Baumgartner, has been withdrawn as necessitated by amendment. 

10. Claims 1, 3-20 are pending. Claim 2 has been canceled by Applicant. Claims 1, 14, 16, 19, 20 are 
independent claims. 
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Claim Rejections - 35 USC § 103 
1 1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the 
differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall 
not be negatived by the manner in which the invention was made. 



12. Claims 1, 4-5, 9, 12-14, 16, 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Katseff et al. (hereinafter Katseff), U.S. Patent No. 5,822,537 issued October 1998, in view of Shibata, Y., 
Media synchronization protocols for packet audio-video system on multimedia information networks 
(hereinafter Shibata), IEEE; January 3-6, 1995 pp.594-601. 

In regard to independent claim 1, Katseff teaches an audio and a video stream stored in a file server, 
said streams are a preferred file format for access by a user (Katseff column 5 lines 18-24, column 6 lines 60-67, 
column 7 lines 1-7, column 8 lines 18-22; compare with claim 1 "retrieving a time-based stream and a motion 
data stream at the server, each stream comprising frames of data "). 

Katseff teaches synchronizing audio and video portions of a recorded presentation by placing the frames 
into an audio buffer, and a video buffer (Katseff column 8 lines 60-65, column 9 lines 1-5; compare with claim 
1 "variably buffering one of the time-based data stream and the motion stream to produce two streams having 
synchronized frames "). 

Katseff teaches a multimedia object at a workstation synchronizing the presentation of audio and visual 
mediums (Katseff column 9 lines 1-5; compare with claim 1 "using the synchronized frames at the client for 
playback of synchronized motion and time-based data to a user. "). 

Katseff does not specifically teach a method whereby the variably buffering occurs at server. However, 
Shibata a rate control message sent to a server, said server dynamically updating the current frame rate, 
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including its parameters (Shibata pages. 595-596 section "System Architecture", and page 596, section "Frame 
rate control"; compare with claim 1, "at the server. It would have been obvious to one of ordinary skill in the 
art at the time of the invention to apply the server participation method of Shibata to the buffer method of 
Katseff, because of Shibata' s taught advantage of server interaction, providing increased equitable workload to 
the client buffer method as taught by Katseff. 



In regard to dependent claim 4, Katseff teaches an audio and a video stream stored in a file server, 
said streams are a preferred file format for access by a user, and said files transferred from server to a 
workstation client (Katseff column 5 lines 18-24, column 6 lines 60-67, column 7 lines 1-7, column 8 lines 18- 
22, Figure 1). 

Katseff does not specifically teach transfer of data values for a frame occuring only for a frame that has 
changed since a last frame was transmitted However, Shibata discloses a method whereby audio data is sent 
from the video server to the client station only during a "talk spurt", with constant frame rates occurring during 
periods of audio silence (Shibata p. 5 97, section 4.3 "Silence detected synchronization"). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply the method of Shibata to the 
method of Katseff, because of Shibata' s taught advantage of selected data transmission, providing increased file 
size and transmission reduction of the audio and video buffering method as disclosed by Katseff. 

In regard to dependent claim 5, Katseff teaches storing and distributing multimedia objects over a 
network (Katseff, column 2 lines 8-12). Katseff does not specifically teach a method of using the Internet for 
object distribution, as claimed However, Katseff teaches an internet as an example of suggested types of 
networks that can be used (Katseff, column 3 lines 58-63), providing the claimed equivalent of the Internet. It 
would have been obvious to one of ordinary skill in the art at the time of the invention to interpret Katseff as 
comprise the Internet for object transmission, providing increased communication range to the distribution 
method disclosed by Katseff. 
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In regard to dependent claim 9, Katseff teaches sending a retrieval message over a network to a 
storage and retrieval system, said storage and retrieval system accessing a file server for file retrieval (preferably 
JPEG Movie Format), and sending the file to the user (Katseff column 6 lines 60-67, column 7 lines 1-7, 
column 8 lines 40-50). 

In regard to dependent claim 12, claim 12 incorporates substantially similar subject matter as claimed 
in claim 1, and in further view of the following, is rejected along the same rationale. 

Claim 12 "wherein the time-based data is a pre-recorded audio track... " is rejected using the same 
arguments presented in the rejection of claim 1 . 

Claim 12 "...and the method further includes synchronizing playback of the pre-recorded audio track" 
and "buffering of the pre-recorded audio track... " is rejected using the same arguments presented in the 
rejection of claim 1. 

Claim 12 "...to allow for coupling with motion data generated in time with the playback of the pre- 
recorded audio track" is rejected using the same arguments presented in the rejection of claim 1 . 

Katseff does not specifically teach implementing the subject matter mentioned in the above three 
paragraphs at a server. However, Shibata teaches a frame rate control method implemented on both the client 
and server-side of a network, whereby a client rate control message is sent to the server, at which time the server 
adjusts its own frame rate (Shibata, pages.595-596 section "System Architecture", and page 596 section "Frame 
rate control"; compare with claim 12 "...at the server... "). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to apply the server method of Shibata to the methods of Katseff, because of 
Shibata's taught advantage of server control, providing increased equitable workload to the video method as 
disclosed by Katseff. 

In regard to dependent claim 13, claim 13 incorporates substantially similar subject matter as claimed 
in claim 1, and in further view of the following, is rejected along the same rationale. 
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Katseff does not specifically teach sequencing output for ordered playback of the synchronized frames, 
as claimed. However, Katseff teaches forward and reverse playback of videos disclosed as examples of videos 
resulting from the implementation of the invention described by Katseff (Katseff, column 14 lines 13-19), 
providing the claimed equivalent of ordered playback. It would have been obvious to one of ordinary skill in the 
art at the time of the invention to interpret Katseff as comprising ordered playback, providing increased 
organization to the method disclosed by Katseff. 



In regard to independent claim 14, claim 14 incorporates substantially similar subject matter as 
claimed in claim 4, and in further view of the following, is rejected along the same rationale. 

Katseff teaches an audio and a video stream is stored in a file server, said streams are a preferred file 
format for access by a user, and said files transferred from server to a workstation client (Katseff column 5 lines 
18-24, column 6 lines 60-67, column 7 lines 1-7, column 8 lines 18-22, Figure 1). 

Katseff does not specifically teach storing a last data value for each channel in each frame transmitted 
over the network. However, Shibata teaches a method whereby a rate controller periodically monitors and stores 
in RAM a current frame rate, a number computed by frames/seconds (Shibata page 596 column 1, section 
"Frame rate control"; compare with claim 14 "storing a last data value for each channel in each frame 
transmitted over the network"). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to apply the frame rate monitoring method of Shibata to the method of Katseff, because of Shibata's 
taught advantage of periodic monitoring, providing increased continuous monitoring of the audio and video 
buffering method as disclosed by Katseff. 

Katseff teaches an audio and a video stream stored in a file server, said streams stored as synchronized 
JPEG Movie Files for access by a user, and said files transferred from server to a workstation client (Katseff 
column 5 lines 18-24, column 6 lines 60-67, column 7 lines 1-7, column 8 lines 18-22, Figure 1; compare with 
claim 14 "retrieving new synchronized frames for transmission over the network"). 

Claim 14 "packaging and transmitting over the network only data for channels having changed data 
values s \ is rejected using the same argument and rationale presented for the rejection of claim 4. 
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In regard to independent claim 16, claim 16 reflects the apparatus used to perform the methods 
comprising computer readable instructions as claimed in claim 1, and is rejected along the same rationale. 

In regard to dependent claim 18, claim 18 reflects the apparatus used to perform the methods 
comprising computer readable instructions as claimed in claim 14, and in further view of the following, is 
rejected along the same rationale. 

Katseff does not teach an apparatus to hold data values last transmitted over a network, as claimed. 
However, Katseff teaches secondary storage devices (i.e. disk arrays) disclosed as examples of suggested 
storage devices that can be used to hold data (Katseff, column 4 lines 61-64), providing the claimed equivalent 
of an apparatus to hold data, providing increased data persistence to the audio/video streaming method as 
disclosed by Katseff. (compare with claim 18 "...a storage device. ..for storing data values... "). 

13. Claim 3 is rejected under 35 U.S.C 103(a) as being unpatentable over Katseff, and Shibata as 
applied to claim 1 above, and further in view of Baumgartner et al. (hereinafter Baumgartner), U.S. 
Patent No. 5,642,171 issued June 1997. • 

In regard to dependent claim 3, Katseff does not teach a difference between the delay for the audio 
and motion data stream to determine which stream is faster is calculated. However, Baumgartner teaches a 
method whereby a current video frame number is subtracted from a current audio frame number to determine if 
the audio is too far ahead of the video (Baumgartner column 13 lines 60-67, column 14 lines 1-2). It would 
have been obvious to one of ordinary skill in the art at the time of the invention to apply the difference 
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calculation method of Baumgartner to Katseff, because of Baumgartner's taught advantage of audio/video 
comparison, providing increased time synchronization to the server buffer method disclosed by Katseff. 

14. Claims 6, 7, 10, 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over Katseff and 
Shibata as applied to claims 1 and 16 above, and further in view of Durward et al. (hereinafter Durward), 
U.S. Patent No. 9,950,202 issued September 1999. 

In regard to dependent claim 6, Katseff teaches an audio and a video stream stored in a file server, 
said streams are a preferred file format for access by a user (Katseff Abstract, column 5 lines 18-24, column 6 
lines 60-67, column 7 lines 1-7, column 8 lines 18-22). Katseff does not specifically teach a method whereby 
motion data is mapped to control movement of a virtual figure displayed in a scene at the client. However, 
Durward teaches a method whereby updated positional data from a person's head position sensor is mapped and 
used to determine the position of a virtual being defined for that user, communicating graphical data to the user 
(Durward, column 6 lines 29-32, column 7 lines 12-20). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to apply the virtual position data of Durward to the method of Katseff, 
because of Durward's taught advantage of three dimensional position mapping, providing increased spatial 
imagery to the video stream method as disclosed by Katseff. 

In regard to dependent claim 7, Katseff does not specifically teach using a body suit for generating 
motion data. However, Durward teaches a method of sensing position data with the use of a body suit 
(Durward, column 3 line 25-33). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to apply the body suit method of Durward to the method of Katseff, because of Durward's taught 
advantage of three dimensional physical contour mapping, providing increased spatial image accuracy to the 
video stream method as disclosed by Katseff. 
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In regard to dependent claim 10, Katseff does not specifically teach using voice data as time-based 
data. However, Durward teaches a method of assigning sound data as voice data (Durward, column 7 line 53- 
57). It would have been obvious to one of ordinary skill in the art at the time of the invention to apply the voice 
data method of Durward to the method of Katseff, because of Durward 5 s taught advantage of voice mapping, 
providing increased aural communication to the audio method as disclosed by Katseff. 



In regard to dependent claim 17, Katseff does not specifically teach an apparatus whereby 
synchronized motion/time-based data is milticasted to clients in a network. However, Durward teaches an 
apparatus whereby users log on to a system via modems, said users data is shared with other users and 
continuously updated so as to create a virtual reality world (Durwald, column 8 lines 22-24, 41-64, Figure 1). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the data sharing 
method of Durward to the method of Katseff, because of Durward' s taught advantage of consistent data sharing, 
providing continuous updating and increased uniform user awareness in a virtual environment to the method as 
disclosed by Katseff. 

15. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Katseff, in view of 
Durward. 

In regard to independent claim 19, Katseff teaches an audio and a video stream stored in a file server, 
said streams are a preferred file format for access by a user (Katseff Abstract, column 5 lines 18-24, column 6 
lines 60-67, column 7 lines 1-7, column 8 lines 18-22). Katseff does not specifically teach a method whereby 
motion data is mapped to control movement of a virtual figure displayed in a scene at the client. However, 
Durward teaches a method whereby updated positional data from a person's head position sensor is mapped and 
used to determine the position of a virtual being defined for that user, communicating graphical data to the user 
(Durward, column 6 lines 29-32, column 7 lines 12-20; compare with claim 6). It would have been obvious to 
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one of ordinary skill in the art at the time of the invention to apply the virtual position data of Durward to the 
method of Katseff, because of Durward's taught advantage of three dimensional position mapping, providing 
increased spatial imagery to the video stream method as disclosed by Katseff. 

Katseff teaches a method whereby a multimedia object at a workstation synchronizes the presentation 
of audio and visual mediums (Katseff column 9 lines 1-5). Katseff does not specifically teach a method of 
playing back in synchronization with movement of the virtual figure the time-based data. However, Durwald 
teaches a method of assigning sound data to a corresponding virtual space, i.e. the assignment of a user's voice 
to the voice of the corresponding virtual being (Durward, column 7 lines 53-60; compare with claim 19 
"playing back in synchronization with movement of the virtual being the time-based data, It would have been 
obvious to one of ordinary skill in the art at the time of the invention to apply the voice data method of Durward 
to the method of Katseff, because of Durward's taught advantage of voice mapping, providing increased aural 
communication to the video method as disclosed by Katseff. 

16. Claims 8, 11, 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over Katseff, and 
Shibata, as applied to claim 1 above, and further in view of Wasserman et al. (hereinafter Wasserman), 
U.S. Patent No. 5,812,791 issued September 1998. 

In regard to dependent claim 8, Katseff does not specifically teach motion data including background 
data used for producing a scene at the server. However, Wasserman teaches a method whereby images are 
decompressed for the use of textures, or backgrounds with overlays of moving video on the still images 
(Wasserman, column 6 lines 8-16). It would have been obvious to one of ordinary skill in the art at the time of 
the invention to apply the background texture method of Wasserman to the method of Katseff, because of 
Wasserman's taught advantage of scenery data inclusion, providing increased visual realism to the video method 
as disclosed by Katseff. 
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In regard to dependent claim 11, Katseff teaches an audio and a video stream stored in a file server, 
said streams are a preferred synchronized file format (JPEG Movie File) for access by a user (Katseff column 5 
lines 18-24, column 6 lines 60-67, column 7 lines 1-7, column 8 lines 18-22; compare with claim 11 "...the 
synchronized data frames include one or more data channels... "). 

Katseff does not specifically teach a descriptor packet transmitted between synchronized frames so that 
a client may join in progress a multicast of synchronized data frames. However, Wasserman teaches a method of 
inserting presentation timestamps into an MPEG stream every 0.7 seconds, said time stamps represent times at 
which a video picture is to be displayed, or starting audio playback times (Wasserman, column 4 lines 37-44; 
compare with claim 1 1 "the server transmitting .... of synchronized data frames. "). It would have been obvious 
to one of ordinary skill in the art at the time of the invention to apply the timestamp method of Wasserman to the 
method of Katseff, because of Wasserman' s taught advantage of embedded synchronization, providing 
increased client synchronization accuracy to the video method as disclosed by Katseff. 

In regard to dependent claim 15, claim 15 incorporates substantially similar subject matter as claimed 
in claim 1 1, and is rejected along the same rationale. 

17. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Katseff, Shi bat a, and 
Durward, as applied to claims 10 and 17 above, and further in view of Baumgartner. 

In regard to independent claim 20, claim 20 incorporates substantially similar subject matter as 
claimed in claims 1, 10, 16 and 17, and in further view of the following, is rejected along the same rationale. 

Claim 20 "retrieving an audio stream... .including frames of data" is rejected using the same arguments 
presented in the rejection of claims 1 and 10. 
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Katseff teaches a method whereby an audio and a video stream is stored in a file server, said streams are 
a preferred file format for access by a user (Katseff column 5 lines 18-24, column 6 lines 60-67, column 7 lines 
1-7, column 8 lines 18-22). 

Katseff does not specifically teach a method whereby a delay is calculated through a server for a frame 
of data on each of the streams. However, Baumgartner teaches a method whereby an audio frame number is 
determined from a query of the corresponding audio frame position, said frame number is used in reference to a 
starting time index (Baumgartner column 6 lines 45-50, column 13 lines 48-59; compare with claim 20 
"calculating a delay. . . " and "for a frame of data on each of the streams "). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to apply the frame number method of Baumgartner to the 
method disclosed by Katseff, because of Baumgartner' s taught advantage of time/frame comparison, providing 
increased time synchronization to the server buffer method disclosed by Katseff 

Claim 20 "calculating a difference between the delay... .of the two streams is faster" is rejected using the 
same argument presented for the rejection of claim 3. 

Additionally, with reference to paragraph 3 of the present rejection, Katseff does not specifically teach a 
method whereby the faster stream is buffered, resulting in synchronized streams. However, Baumgartner teaches 
a method whereby audio is adjusted if it is determined that the audio is too far ahead of the video, or the video is 
adjusted if the video is too far ahead of the audio (Baumgartner column 14 lines 4-7, 15-17; compare with claim 
20 "variably buffering a faster of the streams... .having synchronized data frames"). It would have been obvious 
to one of ordinary skill in the art at the time of the invention to apply the comparison/adjustment method of 
Baumgartner to the method disclosed by Katseff, because of Baumgartner' s taught advantage of audio/video 
speed comparison, providing increased time synchronization to the server buffer method disclosed by Katseff. 

Claim 20 "packaging the synchronized data frames" is rejected using the same argument presented for 
the rejection of claim 16. 

Claim 20 "multicasting the synchronized data frames to one or more clients on a network" is rejected 
using the same argument presented for the rejection of claim 17. 
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Claim 20 u at each client computer... .for display to a user" is rejected using the same argument 
presented in the rejection of claim 1. 

Katseff does not specifically teach a method whereby the variably buffering occurs at server. However, 
Shibata a rate control message sent to a server, said server dynamically updating the current frame rate, 
including its parameters (Shibata pages.595-596 section "System Architecture", and page 596, section "Frame 
rate control"; compare with claim 20, (< at a server computer. It would have been obvious to one of ordinary 
skill in the art at the time of the invention to apply the server participation method of Shibata to the buffer 
method of Katseff, because of Shibata's taught advantage of server interaction, providing increased equitable 
workload to the client buffer method as taught by Katseff. 



18. Applicant's arguments filed 6/18/2003 have been fully and carefully considered but they are not 
persuasive. 

Applicant argues on page 9-10 (also repeated on various pages) of the amendment, that Katseff does not 
disclose the limitations of claim 1 (i.e. accessing of separate audio (time-based) and video (motion-based) data, 
etc.). The examiner respectfully notes that a JPEG movie file interleaves a frame of audio with a frame of video 
(which can be interpreted as a movie separated into separate audio and video component frames) (Katseff 
column 6 lines 64-67). In addition, Katseff also discloses prefetching of both audio data and video data, each 
placed into respective audio and video buffers (Katseff column 8 lines 60-67). 

Applicant argues on page 10 of the amendment that Katseff fails to diclose retrieving a time- based data 
stream and a motion-based data stream at the server, each stream comprising frames of data, as well as 
synchronization prior to receipt at a workstation. The examiner respectfully notes that in addition to the above, 
Katseff discloses an audio and a video stream stored in a file server, said streams are a preferred file format for 
access by a user It is respectfully submitted that audio data is interpreted as time-based data, and video data is 



Response to Arguments 
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interpreted as motion-based data. Shibata teaches rate adjustment at a server (see also Sibata pages 596-597 
Figures 1 and 2, which discloses (frame) rate controllers at both a server and at a client). 

Applicant argues on page 1 1 of the amendment that the claimed invention claims variably buffering one 
of the time-based data stream and the motion stream to produce two streams having synchronized frames. The 
examiner notes that KatsefFs disclosure describes a method for relieving network congestion by monitoring 
buffers' threshold and compensating by reducing video transmittal rate, then reducing audio playback rate 
(Katseff- title). 

Applicant's arguments on pages 12-13, regarding claim 16 are substantially similar to those already 
presented, and have been previously discussed. 

Applicant argues on pages 14-15 of the amendment that Shibata does not suggest audio and video rate 
values be controlled. The examiner respectfully notes that Shibata teaches a rate controller that periodically 
monitors and stores in RAM a current frame rate, a number computed by frames/seconds. The examiner applies 
this teaching (data frame rate control) to the audio and video data of Katseff. 

Applicant argues on page 16 of the amendment that Shibata does not teach comparing values (claim 14). 
The examiner respectfully notes that Shibata's video rate adjustments are the result of data comparisons. 

Applicant's arguments on pages 17-23 are substantially similar to those previously presented above. 

Conclusion 

19. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth 
in37CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the 
mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this 
final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no 
event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this 
final action. 

20. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to William Bashore whose telephone number is (703) 308-5807. The examiner can normally be 
reached Monday through Friday from 1 1 :30 AM to 8:00 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Joseph 
Feild, can be reached on (703) 305-9792. 

Any inquiry of a general nature or relating to the status of this application should be directed to the 
Group receptionist whose telephone number is (703) 305-3900. 

21. Any response to this action should be mailed to: 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 



or faxed to: 



(703) 746-7239 (for formal communications intended for entry) 



or: 



(703) 746-7240 (for informal or draft communications, please label 
"PROPOSED" or "DRAFT 5 ') 



or: 



(703) 746-7238 (for after-final communications) 



Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Fourth Floor (Receptionist). 



William L. Bashore 
September 2, 2003 



SANJIV SHAH 
PRIMARY EXAMINER 



